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14.  ABSTRACT 

Funds  provided  by  this  grant  allowed  us  to  purchase  several  large  compute-servers,  which  supported  intensive  experimentation  conducted  under  the 
auspices  of  MURI FA9550-07- 1-0532.  We  conducted  extensive  mutational  robustness  studies.  Given  a  population  of  variant  programs,  created 
from  an  original  program  by  applying  random  mutations,  we  measure  which  variants  still  pass  all  available  test  cases  and  call  them  "neutral."  The 
fraction  of  all  variants  that  are  neutral  is  defined  as  the  program's  "mutational  robustness."  Even  when  the  mutations  are  restricted  to  statements 
executed  by  the  test  cases,  mutational  robustness  is  surprisingly  high,  36.75\%  on  a  corpus  of  programs  taken  from  22  production  software  projects, 
the  Siemens  benchmark  suite,  and  a  few  specially  constructed  programs.  Next,  we  conducted  experiments  on  the  assembly-level  automated 
program  repair  algorithm  using  a  benchmark  test  suite  of  ten  open  source  programs.  The  ASM  repair  method  successfully  repaired  all  of  the  ten 
programs  on  at  least  one  run,  with  an  overall  success  rate  of  0.54  per  trial.  We  also  conducted  experiments  on  a  recent  extension  of  the  automated 
program  repair  framework  to  elf  files.  This  extension  would  be  useful,  for  example,  when  working  on  embedded  devices  (continued  on  next  page) 
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14.  Abstract  (Continued) 


and  either  the  source  code  is  unavailable  or  there  isn’t  room  to  store  the  complete  tool  chain  required 
for  repairs  at  the  abstract  syntax  tree  level.  We  used  the  DURIP-supplied  computers  to  conduct  many  of 
the  elf  level  experiments,  discovering  that  on  the  same  benchmark  set  that  we  can  repair  five  out  of  the 
ten  programs  at  the  elf  level  without  any  access  to  the  source  code. 
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Under  the  auspices  of  this  grant,  we  purchased  the  following  equipment: 

1.  1  Dell  224-8310  Power  Edge  R815 

2.  1  Dell  224-9502  Power  Vault  NX3000 

3.  1  Dell  224-8310  Power  Edge  R815 

4.  lApple  Z0M4  -  Mac  Pro  Two  6-Core  Intel  Xeon 

5.  1  Dell  224-8310  Power  Edge  R815 

6.  2  Apple  27”  Monitors 

7.  Computer  parts  from  New  Egg(eg.  video  card,  Cooler  Master) 

8.  Video  cards  from  New  Egg 

9.  Video  cards  from  New  Egg 

This  equipment  has  been  used  to  further  the  goals  of  our  MURI-supported  Helix  project.  Our 
focus  has  been  on  developing  methods  for  automatically  repairing  programs  and  studying  muta¬ 
tional  robustness.  Our  method  uses  genetic  programming  to  automatically  search  for  repairs  to 
documented  bugs  in  programs  with  test  cases.  In  earlier  work  we  demonstrated  the  method  on 
abstract  syntax  trees  derived  from  C  source  code.  Over  the  past  year,  we  conducted  experiments 
at  the  assembly-level  using  a  benchmark  test  suite  of  ten  open  source  programs.  The  ASM  repair 
method  successfully  repaired  all  of  the  ten  programs  on  at  least  one  run,  with  an  overall  success  rate 
of  0.54  per  trial.  Using  this  advance  we  demonstrated  repairs  of  programs  compiled  to  assembly 
code  from  several  different  languages,  a  major  extension  of  our  earlier  work. 

We  also  conducted  experiments  on  a  recent  extension  of  the  automated  program  repair  framework 
to  elf  files.  This  extension  would  be  useful,  for  example,  when  working  on  embedded  devices  and 
either  the  source  code  is  unavailable  or  there  isn’t  room  to  store  the  complete  tool  chain  required 
for  repairs  at  the  abstract  syntax  tree  level.  We  used  the  DURIP-supplied  computers  to  conduct 
many  of  the  elf  level  experiments,  discovering  that  on  the  same  benchmark  set  that  we  can  repair 
five  out  of  the  ten  programs  at  the  elf  level  without  any  access  to  the  source  code. 

Finally,  we  conducted  extensive  experiments  on  mutational  robustness  at  all  three  program  repre¬ 
sentation  levels:  abstract  syntax  tree,  assembly,  elf  format  binaries.  Given  a  population  of  variant 
programs,  created  from  an  original  program  by  applying  random  mutations,  we  measure  which 
variants  still  pass  all  available  test  cases  and  call  them  ’’neutral.”  The  fraction  of  all  variants  that 
are  neutral  is  defined  as  the  program’s  ’’mutational  robustness.”  Even  when  the  mutations  are  re¬ 
stricted  to  statements  executed  by  the  test  cases,  mutational  robustness  is  surprisingly  high,  36.75% 
on  a  corpus  of  programs  taken  from  22  production  software  projects,  the  Siemens  benchmark  suite, 
and  a  few  specially  constructed  programs. 

In  addition  to  the  published  papers  documented  on  the  form,  we  have  submitted  the  results 
of  these  experiments  for  publication  at  the  International  Conference  on  Software  Engineering.  We 


are  also  preparing  a  paper  on  the  assembly  and  elf  levels  of  representation  for  submission  either  to 
PLDI  (programming  language  design  and  implementation)  or  ISSTA  (International  Symposium  on 
Software  Testing  and  Analysis,  ) 


